Skip to content

BUG: Issue in the gbq module when authenticating on remote servers #8489 #11141

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

parthea
Copy link
Contributor

@parthea parthea commented Sep 18, 2015

Resolve issue where gbq authentication on remote servers fails silently.

@parthea
Copy link
Contributor Author

parthea commented Sep 18, 2015

Travis is green

@@ -133,7 +132,9 @@ def get_credentials(self):
credentials = storage.get()

if credentials is None or credentials.invalid or self.reauth:
credentials = run_flow(flow, storage, argparser.parse_args([]))
print('Go to the following link in your browser: {0}'.format(flow.step1_get_authorize_url()))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not a big fan of this. I would simply raise with an intelligble message.

@jreback jreback added Google I/O Error Reporting Incorrect or improved errors from pandas labels Sep 20, 2015
@jreback
Copy link
Contributor

jreback commented Oct 9, 2015

@parthea can you update, I think should just raise an informative message

@jreback jreback added this to the 0.17.1 milestone Oct 9, 2015
@parthea parthea closed this Oct 11, 2015
@parthea parthea force-pushed the bq-auth-on-remote-servers branch from 114aab5 to 99bf170 Compare October 11, 2015 13:06
@parthea
Copy link
Contributor Author

parthea commented Oct 11, 2015

@jreback

I'm still working on this PR. I removed my last commit, which automatically closed the pull request. It should open when I submit a new commit.

I want to provide users with a simple way to create the required bigquery_credentials.dat file. Can I add a new function gbq.authorize() to do this?

Then I can raise an informative error message if the credentials are missing or invalid which tells the user to first call the gbq.authorize() function. When users call the gbq.authorize() function, the following message will appear: 'Please visit the following url to obtain an authorization code:' Users then will visit the web page to get the authorization code, and then paste it in.

@jreback
Copy link
Contributor

jreback commented Oct 14, 2015

@parthea .authorize sounds reasonable

@parthea
Copy link
Contributor Author

parthea commented Oct 15, 2015

Add gbq.authorize() method

@parthea parthea reopened this Oct 15, 2015
@parthea parthea force-pushed the bq-auth-on-remote-servers branch 6 times, most recently from ac915d6 to c0d5ec9 Compare October 16, 2015 12:29
@parthea
Copy link
Contributor Author

parthea commented Oct 16, 2015

All tests passed in my local environment. Waiting on travis.

nosetests test_gbq.py -v
test_should_be_able_to_get_a_bigquery_service (pandas.io.tests.test_gbq.TestGBQConnectorIntegration) ... ok
test_should_be_able_to_get_results_from_query (pandas.io.tests.test_gbq.TestGBQConnectorIntegration) ... ok
test_should_be_able_to_get_schema_from_query (pandas.io.tests.test_gbq.TestGBQConnectorIntegration) ... ok
test_should_be_able_to_get_valid_credentials (pandas.io.tests.test_gbq.TestGBQConnectorIntegration) ... ok
test_should_be_able_to_make_a_connector (pandas.io.tests.test_gbq.TestGBQConnectorIntegration) ... ok
test_bad_project_id (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_bad_table_name (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_column_order (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_column_order_plus_index (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_download_dataset_larger_than_200k_rows (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_index_column (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_malformed_query (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_arbitrary_timestamp (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_empty_strings (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_false_boolean (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_null_boolean (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_null_floats (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_null_integers (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_null_strings (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_null_timestamp (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_timestamp_unix_epoch (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_true_boolean (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_valid_floats (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_valid_integers (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_valid_strings (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_unicode_string_conversion_and_normalization (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_zero_rows (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_read_gbq_with_no_project_id_given_should_fail (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_should_return_bigquery_booleans_as_python_booleans (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_should_return_bigquery_floats_as_python_floats (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_should_return_bigquery_integers_as_python_floats (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_should_return_bigquery_strings_as_python_strings (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_should_return_bigquery_timestamps_as_numpy_datetime (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_that_parse_data_works_properly (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_to_gbq_should_fail_if_invalid_table_name_passed (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_to_gbq_with_no_project_id_given_should_fail (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_create_dataset (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_create_table (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_dataset_does_not_exist (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_dataset_exists (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_delete_dataset (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_delete_table (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_generate_schema (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_google_upload_errors_should_raise_exception (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_list_dataset (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_list_table (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_list_table_zero_results (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_table_does_not_exist (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_upload_data (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_upload_data_if_table_exists_append (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_upload_data_if_table_exists_fail (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_upload_data_if_table_exists_replace (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
pandas.io.tests.test_gbq.test_generate_bq_schema_deprecated ... ok

----------------------------------------------------------------------
Ran 53 tests in 357.736s

OK

@parthea
Copy link
Contributor Author

parthea commented Oct 16, 2015

Ready for review. Travis is green.

@parthea parthea force-pushed the bq-auth-on-remote-servers branch from c0d5ec9 to cd3221b Compare October 16, 2015 20:16
@parthea parthea force-pushed the bq-auth-on-remote-servers branch from cd3221b to 55764ea Compare October 16, 2015 21:03
@@ -128,23 +130,44 @@ def test_google_api_imports(self):
except ImportError as e:
raise ImportError("Missing module required for Google BigQuery support: {0}".format(str(e)))

def get_credentials(self):
def authorize(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, you must have the credentials to pass in. pandas shouldn't be responsiblet for saving/retriveing these. .authorize should be almost a trivial call

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I think #11335 may be a better solution since it allows the user to pass in a json key. I'm going to close this PR based on the discussion in #8489

@parthea parthea closed this Oct 16, 2015
@parthea parthea deleted the bq-auth-on-remote-servers branch October 16, 2015 23:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Error Reporting Incorrect or improved errors from pandas
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants